For many tasks such as testing and improvement of network performance, testing, development and improvement of network protocols and debugging problems related to connectivity, performance and protocol issues, it is necessary to capture the data passing through a computer network for later analysis. This general technique is known as packet tracing.
Data transmitted over a computer network is generated at a first node in a network and is received at a second node in the network. After being generated by an application at the first node the data is not usually in a form suitable for transmission over the network. The generated data usually therefore passes through a series of network modules, known as layers of a protocol stack. Each series of network module alters the form of the data, from the form in which it was generated, to a form suitable for transmission over the network. At the second node, the transmitted data is again altered by network modules, from the form in which it was transmitted, to a form suitable for receipt by an application at the second node.
Data is transmitted over a computer network as a series of discrete packets. Capture of data is affected by intercepting the packets as they move from the generating application at the first node to the receiving application at the second node, and recording the data contained in the packets.
Methods are known for capturing data in the form in which it is transmitted over the network, that is, after the generated data has passed through all network modules at the first node, and before it has passed through any network modules at the second node. However, it is also useful to capture data while it is transmitted between the protocol stack layers.
Known methods of capturing data in the form in which it is transmitted over the network include traffic monitor programs such as tcpdump, developed for use in a UNIX operating system environment, windump, developed for use in a Windows operating system environment, and others similar tools such as snoop and ethereal. These tools capture data at the point between the network modules and the network, that is, the data entering and leaving the network. They are unable to capture the data within the protocol stack. Therefore, they are unable to provide information relating to the operation and performance of the network modules constituting the protocol stack.
A known packet tracing system which is capable of capturing while it is transmitted between the protocol stack layers is the Monitor for Application-Generated Network Traffic (MAGNeT). This system is able to monitor data throughout the series of network modules as well as data entering and leaving the network.
The development of high speed computer networks has given rise to greater quantities of data passing through computer networks at higher speeds. In order to capture this data, it is necessary for packet tracing systems to intercept the data packets and record the relevant data in persistent memory at a sufficient rate, so that data is not lost.
The ability to intercept and record the relevant data in high speed computer networks is limited by the availability and method of use of computer memory and processor resources. In general, packets intercepted by the packet tracing system are firstly placed into memory buffers in non-persistent memory and are secondly transferred from non-persistent memory and written to persistent memory. There is a limited amount of space in the memory buffers in non-persistent memory. If the memory buffers are full of packet data when a further packet is intercepted, data will be lost. Therefore, in order to reduce loss of data, the speed at which data is transferred from non-persistent memory to persistent memory, thereby emptying the memory buffers, so that later packets intercepted can be placed into the buffers, is important.
Known packet tracing systems including tcpdump and MAGNeT place the captured packets into memory buffers. These are implemented in non-persistent memory, in computer operating system kernel space memory. The contents of the buffers are then mapped to user space memory before being written to persistent memory. This mapping to user space memory has several disadvantages. It creates additional demand on system resources, especially processing resources; it introduces overhead time involved for the scheduling of processing resources and memory writes which are requested by the user application; and it introduces the need for a context switch for each memory write call that is made by the user application. These disadvantages result in a higher time taken for data to be written to persistent memory, thereby increasing the potential for loss of data.
The potential for loss of data is further increased in systems such as tcpdump where each packet intercepted by the packet tracing system is dealt with individually. Each time data is copied from memory buffers in kernel space memory to user space memory and each time a memory write request is made by a user application, a fixed amount of overhead time is required, independent of the amount of data being dealt with. Where one packet is copied at a time, this overhead is incurred for each packet, reducing the rate at which the data can be transferred from memory buffers to persistent memory, thereby increasing the potential for loss of data.
The packet tracing system MAGNeT has several features to reduce loss of data. The system uses a fixed size circular buffer in kernel space memory which has a series of slots to each of which a single data packet can be written. MAGNeT also has the ability to aggregate multiple packets and map these in bulk to user space which reduces the overhead time involved in mapping individual packets. However, the reduction in overhead time depends on the frequency with which MAGNeT maps data in user memory space. MAGNeT performs this mapping periodically. If the period Is too short, the number of packets that are aggregated will be less than optimal and therefore the reduction in the overhead time involved in the transfer will also be less than optimal. Conversely, if the period is too long, the buffers in kernel memory will become full, and further intercepted packets will not be able to be stored in the buffers, resulting in loss of data. MAGNeT is unable to ensure that the optimal period will be used.
Yet another challenge for packet tracing systems is the desire to capture only a portion of the data passing through the network. Methods for capturing all data can result in excessive amounts of information, of which only a small portion is needed or is useful for analysis. Thus, the large body of data must be mined to find the relevant information. In many situations, such as the diagnosis of specific network/protocol stack problems, it is useful to be able to capture only the data which is relevant to the specific analysis task. For example, in order to diagnose a specific problem, it may be useful to capture data which has passed through the network immediately before or after the occurrence of the specific problem. In order to achieve this, the user of the packet tracing system may wish to start and stop the capture of data at various times, or may wish for the capture of data to automatically start and stop in response to certain defined events.
Traffic monitor programs such as tcpdump can be used to restrict the data that is captured to a limited degree. For example, parameters may be set within tcpdump such that only a certain type of data is captured (for example, only data received from a particular computer). However, while the program is running, tcpdump will capture all data specified by the parameters set. There is no capability within tcpdump to begin capturing data in response to a user request or a particular event itself.
Similarly, MAGNeT is unable to record a subset of the data passing through the network. In particular, MAGNeT is unable to record a subset of data in response to a user request or a particular event.
It is an object of the invention to provide a packet tracing system and method for capturing data in a computer network, which overcomes or at least mitigate some of the abovementioned limitations of known packet tracing systems and methods.
It is a further object of the invention to provide improved packet tracing capability in high speed computer networks.
It is a further object of the invention to provide packet tracing in response to a user request or a particular event, in order to capture a portion of data passing through a computer network.